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1.0  INTRODUCTION 


This  report  reviews  ALPHATECH  progress  to  date  in  developing  an  Integrated 
Analysis  Technique  (IAT).  Particular  emphasis  has  been  placed  on  technical 
capabilities  developed  during  FY84,  and  their  role  within  the  context  of  the 
current  three-year  development  period  (FY83-85). 

Development  thus  far  has  produced  the  component  techniques  and  notational 
formats  that  together  will  comprise  a  comprehensive  analysis  and  design  tool. 
In  order  to  see  why  these  components  have  been  developed  and  how  they  can  be 
integrated  effectively,  we  shall  first  review  the  scope  and  background  of  the 
current  development  effort  [Sections  1.1  -  1.2],  After  identifying  what  types 
of  methods  are  required,  and  why,  we  outline  the  technical  approach  that 
ALPHATECH  has  taken  in  developing  IAT  [Section  1.3).  Finally,  we  describe  the 
current  status  of  methods  development  by  summarizing  technical  accomplishments 
to  date  and  showing  where  these  achievements  fit  within  the  current  develop¬ 
ment  effort  [Section  1.3]. 

Technical  capabilities  developed  thus  far  are  presented  in  more  detail  in 
Section  2.0.  Section  3.0  describes  a  preliminary  validation  of  IAT  methods, 
based  on  those  IAT  components  and  techniques  that  have  been  developed  during 
FY83-84.  Section  4  re-examines  development  status  and  identifies  work 
required  during  FY85  to  insure  the  integration  of  all  IAT  components  and 
their  applicability  to  the  analysis  of  C3  systems  performance. 


1.1  SCOPE  AND  RATIONALE 

1.1.1  Objectives  for  Developing  IAT: 

Why  IAT  is  needed,  and  what  requirements  it  must  meet. 


/  IAT  is  being  developed  to  provide  quantitative  information  about 
C3,  human/system  performance.  This  information  can  then  be  used 
to  guide  the  analysis  and  design  of  new  systems  as  well  as  the 
improvement  or  reconfiguration  of  existing  ones. 


IAT  as  an  integrated  methodology 

To  meet  these  objectives,  IAT  must  provide  techniques  to  predict  and  evaluate 
human/ system  behavior.  Procedures  are  needed  for  analyzing  and  representing 
the  dynamics  of  system  behavior  (in  performance  models).  Because  performance 
modeling  depends  on  how  well  system  structure  can  be  described  (in  "static” 
models),  techniques  must  also  be  developed  to  analyze  and  display  structural 
elements  and  their  Interrelationships .  In  short,  IAT  must  integrate:  (I) 
descriptive  methods  for  building  structural  and  performance  models;  and  (2) 
predictive  methods ,  which  can  exercise  these  models  to  derive  quantitative 
information.  Decision-makers  and  operations  personnel  will  then  be  able  to 
use  IAT  to  help  answer  questions  about  the  structure  and  behavior  of  manned 
C3  systems. 


IAT  is  also  envisioned  as  an  integrated  methodology  in  terms  of  its  use  for 
systems  engineering  purposes.  In  this  context: 

IAT  as  an  analysis  tool  can  — 

1)  Describe  the  structure  of  C3  man-machine  systems. 

2)  Represent  the  dynamics  of  human/system  behavior. 

3)  Evaluate  and  predict  human/system  performance. 

IAT  as  a  design  tool  can  be  used  to  assess  — 

4)  What  aspects  of  a  system  should  be  improved  and  by  how  much. 
(Information  from  #2  and  3  will  make  it  possible  to  compare 
achieved  performance  with  desired  levels  of  effectiveness, 
efficiency,  and  system  adaptability.) 

3)  What  parts  of  a  system  design  should  be  altered  to  achieve  the 
desired  results.  (The  performance  models  from  #2  can  be  used 
to  run  sensitivity  analyses  once  the  criterion  of  interest  and 
magnitude  of  desired  improvements  have  been  identified.) 


1.1.2  Technical  Motivation 


There  are  several  important  reasons  why  traditional  systems  engineering  and 
engineering  psychology  methods  are  insufficient  for  deriving  predictive 


performance  data  for  C3  systems: 


1)  The  characteristics  of  manned  C3  systems  are  unique  (e.g., 

timing  constraints,  authority/control  in  military  organizations, 
nature  of  decision-making,  mission  objectives). 


2) 


There  is  a  need  for  organizing  descriptive  information  about  C3 
systems  in  a  format  that  facilitates  analyzing  system  structure 
and  modeling  system  performance. 


3)  The  complete  range  of  system  life  cycle  development  stages  must 
be  addressed  in  analyzing  existing  systems  and  designing  new  ones. 


Each  of  these  issues  is  addressed  below. 


Attributes  of  Manned  C3  Systems 


Manned  C3  systems  have  characteristics  that  are  difficult  to  capture 
with  traditional  modeling  tools.  The  activities  associated  with 
assessing  threats  and  reacting  effectively  have  unique  attributes,  in 
terms  of  time  available  (vs.  time  required)  to  make  decisions.  The 
role  of  military  organization  and  communications  connectivity  must 
also  be  analyzed  to  understand  their  effects  on  functions  like  threat 
assessment  and  resource  allocation. 


TradiCional  descriptive  methods  may  indeed  be  used  to  analyze 
physical  composition  (nodes)  and  connectivity,  and  to  display  the 
result  graphically  in  block  diagrams  or  tree  structures  [1,21. 
Functional  analysis  methods  can  also  help  to  trace  the  flow  of  data 
and  processes  (e.g.,  IDEF0,  Operational  Sequence  Diagrams  [1]).  And 
modeling  languages  like  IDEF2  [6]  and  SAINT  [7]  can  represent  system 
dynamics.  However,  none  of  these  methods  has  been  designed  to 
analyze  the  effects  that  organizations,  policies,  plans,  and  goals 
have  on  human  and  system  performance.  These  become  critical  when  a 
C3  system  must  respond  to  attrition  or  destruction  of  its  elements, 
and  must  reallocate  resources  to  maintain  functional  integrity. 

There  remains,  then,  the  need  for  a  descriptive  method  that  can 
capture  aspects  of  human  behavior  within  C3  systems  (including,  for 
example,  decision-making,  goal  setting/evaluation,  span  of  control, 
and  task  performance). 


•  Need  for  Organizing  Descriptive  Information 

If  quantitative  information  is  to  be  used  to  help  improve  systems, 
mechanisms  must  be  in  place  to  insure  traceability  backwards  and 
forwards,  between  high-level  descriptions  of  system  structure  and 
representations  of  human/system  performance.  Analysts  and  planners 
need  to  see  how  quantitative  changes  in  performance  parameters  impact 
specific  tasks,  resources,  decisions,  span  of  control,  and  authority. 
Unless  a  consistent  syntax  and  semantics  can  be  used  as  the  basis  for 
both  static  and  dynamic  analyses,  such  traceability  cannot  be 
guaranteed.  Although  many  modeling  techniques  now  available  provide 
internally  consistent  formalisms,  none  that  have  been  evaluated  thus 
far  [1,2]  permit  the  desired  linkage  between  structural  and 
performance  models  (e.g.,  there  is  no  automatic  means  for  relating 
IDEF0  function  diagrams  with  PERT,  SAINT,  or  IDEF2  rep tesentations) . 


•  System  Life-Cycle  Development 

Few  methodologies,  if  any,  from  the  disciplines  of  systems 
engineering  and  engineering  psychology  can  be  used  throughout  the 
complete  cycle  of  systems  development  —  from  concept  of  operations 
and  needs  analysis  through  design,  implementation,  test,  verification 
and  validation,  user  acceptance,  maintenance  and  support.  This 
limitation  is  a  consequence,  in  part,  of  the  split  between  structural 
analyses ,  which  are  especially  useful  for  front-end  planning  and 
re-design;  and  performance  analyses ,  whose  results  become  most 
important  for  defining  design  alternatives  and  predicting  their 
outcome.  Organizing  descriptive  information  in  the  manner  outlined 
above  for  IAT  will  provide  information  relevant  to  a  greater  portion 
of  the  life-cycle  than  the  portion  covered  by  any  single  descriptive 
or  predictive  technique. 


Figure  1-1  presents  the  limitations  of  representative  systems  engineering 
methods  with  respect  to  life-cycle  applicability  [11]. 
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1.2  DEVELOPMENT  BACKGROUND 


IAT's  development  had  its  origin  in  a  1980  incident  affecting  operations  at 
NCMC  (NORAD  Cheyenne  Mountain  Complex)  MWC  (Missile  Warning  Center).  Because 
of  the  human  factors  involvement  in  that  incident,  AFAMRL  initiated  a  field 
study  to  analyze  function  flow  in  NCMC  operations.  IDEF0  was  used  as  the 
descriptive  analysis  method  [1],  and  its  applicability  to  manned  C3  systems 
was  compared  with  other  systems  engineering  and  engineering  psychology  methods 
[1,2]. 

None  of  the  methods  which  were  evaluated  in  the  FY83  time  frame  was  found 
adequate  for  describing  either  the  structure  or  behavior  of  manned  C3  systems 
such  as  those  at  NCMC.  Requirements  were  defined  for  a  new  method  (1AT)  that 
would  integrate  appropriate  features  from  existing  methods,  yet  go  beyond  the 
capabilities  of  any  single  method  that  was  examined.  In  particular,  IAT  must 
address  human  engineering  as  well  as  system  engineering  aspects  of  C3  system 
performance. 


1.3  DEVELOPMENT  STATUS 

Methods  development  for  IAT  has  followed  a  four-phased  approach: 

1)  Identify  needs  &  requirements  for  IAT. 

2)  Develop  IAT  conceptual  framework. 

3)  Validate  IAT  methods. 

4)  Develop  applications  materials. 

To  date,  work  has  been  completed  on  1.  Portions  of  2  and  3  have  been  com¬ 
pleted  during  FY83-84,  and  are  scheduled  for  completion  during  FY85.*  The 
emphasis  for  FY85,  however,  will  be  on  3  and  4;  viz., 

-  Use  techniques  already  in  place  to  analyze  human/system  behavior 
in  a  real-world  C3  system  (e.g.,  NCMC  MWC). 

-  Develop  guidelines  and  procedures  for  applying  the  techniques 
and  insuring  that  the  structural  and  performance  modeling 
components  of  IAT  are  integrated. 

Figure  1-2  summarizes  the  capabilities  developed  through  FY84  and  specifies 
what  remains  to  be  done  in  FY85.  Further  details  about  each  accomplishment 
appear  below  in  Sections  2  and  3.  Section  4  will  describe  the  approach  to  be 
taken  for  the  FY85  effort. 


♦Preliminary  validation  of  IAT  methods  was  based  on  SIMCOPE-1.  SIMCOPE-1  is 
a  laboratory  real-time  simulation  of  a  strategic  Missile  Warning  Officer's 
crewstation  in  a  hypothethical  Command  Warning  Center  (CWC). 


[  ]  indicate  relevant  documentation,  primary  source 
"TBC"  •  to  be  completed  during  FY85 


11.0  Identify  needs  t  requirements  for  1ATI 
Develop/defined: 

Questions  about  C3  human  a  system 
performance  (that  IAT  must  address) 

C1  system  performance  measures 
useful  In  addressing  questions 

I0EFo  description  of  function  flow 
(NORAD  NCHC/HWC) 

Data  requirements  of  I0EFo  and  other 
systems  analysis  t  design  methods 

Comparative  analysis  and  critique  of 
systems  analysis  4  design  methods,  to 
evaluate  relevancy  for  addressing  C3 
system  performance 

Set  of  desired  capabi lities  for  1AT 


[1,  2  (revised)] 

[1] 

[1] 

[1] 

Cl] 

[1,  2  (revised).  4,  5  (TBC)] 


|2.0  Develop  1AT  Conceptual  Framework  I  [2,  4  (revised)] 

STRUCTURAl  MODELING 

Review  of  systems  engineering  methods 

appl tcable  for  Uf  structural  analyses  [2] 

Formal  specification  of: 

•  Decomposition  process 

•  Dimensions1  for  analyzing  C3  systems  [2,  3  (revised)] 

•  Data  structures  (frames) 

PERFORMANCE  HOOELING 


Rev  tew/ taxonomy  of  analytic  techniques 
relevant  to  I AT  questions  [2] 

Use  of  Queueing  Network  Theory  (QNT)  models  [2  (overview),  5  (detail)] 


QNT  approaches  to  modeling  himian  performance  [5  (preliminary),  TBC] 

Specification  of  QNT  with  respect  to  data, 
functional  usage  requirements  (TBC] 

Dynamic  performance  analysis  from  static 

descriptions  [5  (preliminary),  TBC] 


13.0  Validate  1AT  Methods  I 

S1MC0PE-1:  STRUCTURAL  HOOELING 

•  I0EFo  functional  decomposition 

•  SHOR  decomposition 

•  Frame  representations 

•  Application  of  recursive  formulae  to 

-  10EFo  Decomposition 

-  SHOR  Decomposition 

S1MC0PE-1:  PERFORMANCE  HOOELING 

•  Oata  flow  diagrams  (DeMarco) 

•  QNT  Representation 

•  Use  of  QNT  to  answer  hunan/system 
performance  questions 

NORAD:  STRUCTURAl  i  PERFORMANCE  HOOELING 


C2] 


[1] 

[5.  TBC] 
[TBC] 


14.0  Develop  1AT  Applications  Materials! 

Procedures  t  guidelines  for  applying  IAT 
techniques 

Documentation  (user's  manual ). 

Figure  1-2.  IAT  Developments: 


[2.  3,  5  (prelim.),  TBC  (revised)] 
[2  (planned),  TBC] 

n-nio 

Required  Capabilities 


2.0  METHODS  DEVELOPMENT:  SUMMARIES  OF  CAPABILITIES  DEVELOPED  TO  DATE 


The  Conceptual  Framework:  An  Idealized  Picture  of  IAT  Use 

When  fully  developed,  IAT  will  consist  of  a  five-step  analysis  procedure:* 

1)  Analyze  the  structure  of  the  system  or  subsystem  of  interest  in 
terms  of  GOALS,  ORGANIZATIONS,  PROCESSES,  RESOURCES,  and  their 
constituent  elements;  build  a  structural  model  to  show  the 
relationships  among  elements. 

2)  Extract  data  from  the  model  in  1)  and  represent  the  derived 
information  in  appropriate  formats  (e.g.,  matrices  or  frames). 

3)  Derive  MOPs  (measures  of  performance)  and  MOEs  (measures  of 
effectiveness)  using  the  structural  model  from  1)  and  the  data 
representations  from  2). 

4)  Collect  performance  data  and  construct  a  performance  model 
(e.g.,  using  a  network  of  queues). 

5)  Specify  scenarios,  consisting  of  real-world  events  that  activate 
or  stop  processes  in  the  performance  model  from  4);  exercise  the 
model  using  appropriate  methods  (e.g.,  closed-form  analysis, 
computer  modeling  and  simulation,  empirical  tests  with  human 
subjects  and  mockups). 

Figure  2-1,  below,  illustrates  these  steps  and  their  outputs  or  products. 
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Figure  2-1 .  IAT  Conceptual  Framework  -  FY85  (FEO)* 


*This  procedure  lists  the  logical  set  of  activities  that  an  analyst  would 
carry  out  in  analyzing  an  existing  system  to  derive  quantitative  information 
Steps  1-5  above  are  Idealized  —  in  actual  practice,  an  analyst  must  decide 
which  steps  to  follow,  and  in  what  order,  depending  on  the  complexity  of  the 
system,  difficulty  in  collecting  performance  parameter  data,  data  require¬ 
ments  of  the  analytic  tool,  and  intended  use  of  the  performance  information 

that  is  generated. 
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The  sections  below  treat  Che  techniques  and  formalisms  ALPHATECH  has  developed 
so  far  that  support  each  of  the  five  steps. 

2.1  TECHNIQUES  FOR  BUILDING  STRUCTURAL  MODELS 

2.1.1  Dimensions  for  Analyzing  C3  System  Structure 

Designers  and  planners  need  answers  to  questions  about  system  structure  to 
determine  what  changes  need  be  made  to  improve  human/system  performance.  Four 
perspectives  are  required  for  analyzing  system  structure  in  order  to  answer 
these  questions: 

1)  GOALS  —  capture  the  requirements  or  constraints  that  mission 

objectives  impose. 

2)  ORGANIZATIONS  —  site  of  control  or  authority;  personnel  play 

roles  at  different  levels  of  organization,  and 
have  responsibilities  allocated  to  them. 

3)  PROCESSES  —  activities  carried  out  to  fulfill  goals. 

4)  RESOURCES  —  agents  (man  or  machine);  equipment,  materials, 

physical  layout  of  components  and  their  connections. 

To  answer  specific  questions  and  see  the  relationships  among  1-4,  it  is 
necessary  to  break  down  each  of  these  four  dimensions  into  its  constituent 
elements.  Structured  decomposition  should  proceed  until  specific  tasks  or 
transactions  can  be  Isolated:  "tasks”  and  "transactions"  are  lower-level 
constituents  of  PROCESSES  (i.e.,  subprocesses);  tasks/transactions  are  carried 
out  by  agents  (man  or  machine),  who  have  been  given  responsibility,  resources, 
and  goals  that  they  must  meet. 


2.1.2  Recursive  Decomposition 

Although  many  existing  methods  from  systems  engineering  provide  techniques  to 
handle  decomposition,  none  evaluated  in  [1,2]  treat  the  relationships  among 
GOALS,  ORGANIZATIONS,  PROCESSES,  and  RESOURCES  in  a  systematic  fashion.  IAT 
uses  set  theory  notation  to  describe  levels  of  analyses  within  each  of  these 
four  dimensions;  i.e.,  one  can  keep  track  of  GOALS/subgoals,  PROCESSES/ 
subprocesses,  etc.  And  to  specify  relationships  among  elements  across 
different  dimensions,  a  recursive  approach  is  used. 

Figure  2-2  shows  the  four  dimensions  of  decomposition  and  how  relationships 
among  elements  can  be  described  recursively.  Figure  2-3  suggests  how  matrix 
notation  can  be  used  to  capture  additional  relationships. 
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DIMENSIONS 


A.  DECOMPOSITION  MATRICES  ("consists  of/is  part  of") 


0 


i 

i 


X 

X  o[+1 


GOAL  DECOMPOSITION:  Describes  partitioning  of 
goals  into  subgoals  for  purposes  of  assignment 
(to  organizational  elements  who  are  responsible 
for  meeting  these  subgoals) 


ORGANIZATIONAL  (AUTHORITY)  DECOMPOSITION: 
defines  lines  of  authority  and  responsibility 


X 


PROCESS  DECOMPOSITION:  specifies  constituency 
relations  between  higher-level  processes  and 
their  components  (subprocesses);  describes  what 
subprocesses  are  required  (e.g.,  for  producing 
a  specific  output  at  a  higher  level) 


R 


l 

i 


X 


RESOURCE  DECOMPOSITION  ("FAMILY  TREE"):  defines 
constituency  of  physical  structure;  e.g., 

squadron 
/  \ 
aircraf t 

/  I  \ 

engine  avionics  ... 


B.  DEPENDENCY/CONNECTIVITY  MATRICES  ("connected  to/flows  to/depends  on") 


X 


ORGANIZATIONAL  COORDINATION:  describes  which 
organizational  elements  can  interact  (e.g., 
exchange  information,  inform  or  coord inate-with) 


P 


i 

i 


PROCESS  DEPENDENCY:  expresses  what  is  required 
for  process  input  and  output  (vs.  what  is 
actually  available) 


RESOURCE  CONNECTIVITY:  specifies  which 
resources  can  send/receive  information 
(from  each  other) 


( Continued) 


C.  ASSIGNMENT  MATRICES  ("is  assigned  to") 


GOAL-SETTING  RESPONSIBILITY:  represents 
partitioning  of  goals  or  objectives  among 
organizational  elements 


0 


l 

i 


X 


PROCESS  ASSIGNMENT  RESPONSIBILITY:  assigns 
responsibility  for  processes  to  organizational 
elements  (can  be  used  for  long-range  planning) 


X 


RESOURCE  ASSIGNMENT  RESPONSIBILITY:  reflects 
issues  of  "ownership"  and  specifies  site  of 
control  over  resource  disposition 


0 


l 

i 


X 


ORGANIZATION  ASSIGNMENT  RESPONSIBILITY: 
describes  who  can  delegate  responsibilities  to 
whom 


D.  ASSIGNABILITY  MATRICES  ("can  be  assigned  to/used  for") 


0 


l 

i 


X 


* 


Capable  of  having  goal-setting  responsibility 


0 


l 

i 


X 


* 


Capable  of  taking  responsibility  for  various 
processes 


Where  {  3*  indicates  possibility-of-assignment .  Each  cell  in  these  matrices 

is  an  index  (numerical  quantity)  that  represents  relative  capability;  e.g.,  t 
show  how  well  an  individual  could  perform  a  given  task.  This  notation  can  be 
used  to  specify  capacity-for-change,  or  adaptability  of  a  C3  system  with 
respect  to  its  external  environment. 


2.1.3  Derivation  of  MOPs  (Measures  of  Performance)  and  MOEs  (Measures  of 
Effectiveness)  ~™ 

MOPs  are  associated  with  PROCESSES  and  describe  how  well  a  specific  process  or 
subprocess  has  been  executed  —  using  quantitative  data  to  specify  factors 
like  extent,  duration,  frequency,  and  currency. 

MOEs  are  associated  with  GOALS  and  describe  the  extent  to  which  requirements 
have  been  met;  e.g.,  have  mission  requirements  been  satisfied?  For  C3  systems 
MOEs  might  include  assessments  of  errors  (type,  number),  target  damage, 
efficiency  counts  (response  time,  kill  ratios),  and  survivability  estimates. 

In  terms  of  the  IAT  Conceptual  Framework,  shown  in  Figure  2-1,  MOPs  and  MOEs 
can  be  defined  according  to  levels  and  polnt-of-vlew: 

At  a  given  level  (4),  appropriate  quantitative  measures  (MOPs)  will  describe 
how  well  a  particular  process  (P  )  has  been  carried  out.  The  organizational 

U,  1 

element  (0,)  responsible  for  this  process  will  be  interested  in  these  measures 

^  i  4 

to  determine  whether  the  goal  (G . )has  been  met  at  this  level.  But  since  G . , 

l  i  1  4-1  1 

0^,  and  P  have  all  been  assigned  from  a  higher-level  organization  (0  ), 

other  measures  (MOEs)  will  be  needed  to  assess  the  extent  to  which  goals  at 

4-1 

this  higher  level  (G  )  have  also  been  met. 

Figures  2-4  and  2-5  illustrate  MOPs  and  MOEs  with  respect  to  the  IAT 
Conceptual  Framework.  Figure  2-6  shows  how  matrix  notation  can  be  used 
to  describe  Goal  vs.  MOP  relationships. 


NOW  WEU  IMS 
PROCESS  f\  DONE? 


NOTE:  FNOH  A  TOP- OWN  VIEWPOINT,  AN  NOP  AT  A  HIGHER  LEVEL 
(••9*  •  NOP  IV)  DETERMINES  AN  HOE  AT  THE  NEXT  LOWER 
LEVEL  (HOE  #2).  i.|» 


Figure  2-4.  Relationship  Between  MOPs  and  MOEs  in  IAT 
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Figure  2-5a.  Example  of  "Goal  Hierarchy”  -  Tree  of  Objectives 
(from  Bennett  et  al»,  1979  [6]) 
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Figure  2-5b.  Example  of  Bennett's  "Goal  Hierarchy"  Shown  in  IAT  Framework 
(Top  3  levels  only  from  Figure  2-5a,  above) 


2.1.4  Data  Structures 


Set  theory  and  matrix  notations  have  been  Introduced  In  the  previous  sections 
to  represent  Information  about  system  structure  and  behavior.  These  notations 
should  be  viewed  as  the  theoretical  basis,  or  foundation,  for  step-by-step 
rules  that  will  help  users  collect  data  and  build  structural  and  performance 
models. 

Frame  notation  has  also  been  developed  by  ALPHATECH  for  IAT  applications  [2]. 
Frames  constitute  major  data  sets  for  each  of  the  four  dimensions:  GOALS, 
ORGANIZATIONS,  PROCESSES,  RESOURCES.  “Slots"  describe  the  data  elements  of  a 
frame. 

The  advantages  of  using  frames  and  slots  are  as  follows: 

1)  Relationships  that  might  be  captured  in  several  different 
matrices  can  be  grouped  together  on  a  single  frame. 

2)  Slots  can  be  used  with  and  without  entries  to  indicate  whether 
performance  data  are  or  are  not  available. 

3)  Default  values  can  be  defined  in  slots  for  carrying  out 
sensitivity  analyses  and  "zero-order"  estimates  of  performance. 

4)  Cross-referencing  can  be  handled  by  supplying  pointers  from 
f rame-to-frame  (indicated  as  entries  on  slots). 

5)  Slots  can  be  added  or  deleted  to  specify  attributes  of  the 
dynamic  characteristics  of  a  process. 

6)  The  inherent  nesting  properties  of  frame  and  slot  notation  make 
it  possible  to  capture  information  from  structural  models 
(recursive  decompositions  and  matrices  in  IAT). 

Figure  2-7  illustrates  frame  and  slot  notation. 
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Figure  2-7.  Example  of  IAT  Structural  Description  and  Process  Frame 
for  the  Process  "Monitor  for  Enemy  Missile  Launch” 
(Based  on  SIMCOPE-1  Analysis  [5]) 


2.2  PERFORMANCE  MODELING 


To  generate  estimates  of  human/system  performance,  predictive 
analysis  methods  from  operations  research  and  engineering 
psychology  require  information  about  system  structure  and  system 
behavior. 


Information  about  system  structure  can  be  derived  from  the  hierarchical 
decompositions,  matrices,  and  frames  used  in  the  IAT  structural  modeling 
component.  Information  about  system  behavior  needs  to  be  derived  from  a  level 
of  description  that  captures  the  dynamic  properties  of  human  and  system 
performance.  These  properties  will  be  represented  in  IAT  performance  models. 
Important  properties  of  manned  C3  system  behavior  can  be  described  in  IAT 
performance  models;  e.g., 

1)  Procedural  Operations:  processes  in  the  system  (identified  as 
decomposed  "P's"  in  the  structural  model)  can  be  described  by  a 
sequence  of  steps  to  indicate  the  logic  of  each  operation  (i.e., 
"scripts”) . 

2)  Parallel  Processing:  any  number  of  processes  can  be  activated 
simultaneously. 

3)  Shared  Resources:  some  processes  require  resources  which  are 
contended  for  by  other  resources. 

4)  Operational  Loading:  load  on  a  system  varies  directly  with  the 
number  of  processes  active  and  the  number  waiting  on  resources. 

5)  Process  Communication:  processes  need  to  transfer  data  and/or 
materials  to  other  processes:  transfer  may  also  include 
information  about  the  process  Itself  (i.e.,  copies  of  the 
process  may  be  transferred).  Note  that  actual  transfer  is 
carried  out  by  resources  (specified  by  the  resource 

connectivity  matrix  [R*  x  R  *] ,  Figure  2-3). 


2.2.1  Constraints  on  Building  Performance  Models 

A  performance  model  will  be  usable  only  in  so  far  as  It  represents  Information 
with  the  right  properties  —  "right"  in  this  sense  means  that  the  data  can 
serve  as  input  to  an  analytic  procedure.  Different  procedures  exhibit 
different  data  needs,  in  terms  of  both  structure  and  content.  For  example, 
statistical  techniques  like  regression  require  inputs  that  specify  observa¬ 
tions  of  variables  (x,y);  whereas  goal  programming  methods  require  coefficient 
data  and  goal  values. 


2.2.2  Specification  of  Analytic  Methods 


Since  building  usable  performance  models  requires  knowing  the  data  require¬ 
ments  of  an  analytic  method  or  methods,  the  tools  used  to  carry  out  perfor¬ 
mance  analysis  in  IAT  must  be  well-specified,  with  respect  to  input/output 
requirements.  Work  done  earlier  on  IAT  {l,pp.  1 66 f f . ;  2]  has  addressed  the 
task  of  describing  data  requirements  of  analytic  methods.  The  results  of 
these  studies  show  that  the  types  of  information  l.sted  below  will  be  required 
(minimally)  as  inputs  to  methods  such  as  Queuing  Network  Theory  (QNT): 

1)  Synchronous /asynchronous  relationships  between  processes 
(derived  from  the  IAT  process  dimension). 

2)  Description  of  actual  data  stores  in  the  physical  system 
(derived  from  the  resource  dimension). 

3)  Description  of  the  actual  data  flows  during  event  processing 

1  £+1 

(derived  from  [R^  *  ]  connectivity  matrices,  shown  in 

Figure  2-3). 

2.2.3  QNT  Approaches  to  Modeling  Human/System  Performance 

Studies  conducted  during  FY82-84  have  selected  QNT  as  a  method  appropriate  for 
generating  quantitative  estimates  of  human/system  performance  in  C3  systems 
[1,2}.  QNT  was  selected  because  its  data  outputs  are  time-related  and 
capacity-related ,  and  these  relations  are  critical  for  analysts  and  designers 
to  understand  so  that  they  can  improve  existing  systems  or  design  new  ones. 

However,  traditional  queuing  methods  are  not  designed  to  take  account  of  human 
behavior  at  the  level  of  complexity  exhibited  in  C3  systems.  Modifications 
to  standard  QNT  must  be  made  to  address  factors  like  accuracy  and  error, 
associated  with  human  operators  who  carry  out  specific  tasks  at  workstations. 
Procedures  have  been  developed  during  the  FY83-84  efforts  for  using  QNT  to 
represent  task  flow  and  to  capture  the  impact  that  errors  can  have  on  through¬ 
put  [51.  Figure  2-8  summarizes  the  procedure  to  represent  task  flow. 


1.  Define  any  function  or  process  as  the  processing  of  a 
given  group  of  related  problems,  tasks,  or  jobs  in 
some  allowable  sequence.  Call  these  tasks ,  T. 

2.  Each  task  T  has  a  set  of  attributes  A.  These  attri¬ 
butes  can  be  defined  in  terms  of  a  variable  set  x  and 
a  relationship  set  f(x),  where  x  is  the  set  of  all 
relevant  variables. 

3.  Tasks  can  be  clustered  into  classes  based  on  common 
attributes  via  a  simple  clustering  algorithm. 

4.  Tjj  is  the  i-th  task  belonging  to  the  j-th  class. 

5.  Tjj  has  attributes  Aj^. 

6.  There  are  assignable  resources,  or  “servers”  in  a 

queuing  theory  sense,  (sj,  S2,  *•*,  s jj). 

7.  Each  resource  has  a  maximum  processing  capacity  Cjj 
( for  any  task) . 

8.  Each  resource  processes  a  class-j  task  at  a  rate  ry  jj, 
where  nj ^  is  the  mean  of  an  exponential  distribution. 

9.  Arrivals  are  Poisson,  and  the  arrival  rate  of  class-j 
tasks  is  Ajj  i.e.,  average  time  between  arrivals  of 
Tlj,  T2  j  ,  •••,  Ttj  is  1/  Aj . 

10.  An  external  scenario  drives  the  overall  task  arrival 
rate  A  =  EAj . 

11.  For  simplicity,  assume  that  all  resources  have 
identical  maximum  task  processing  capacities  C, 
i.e.,  C[  =  C2  =  •••  =  Cjj  =  C. 

12.  Assume  that  their  average  task  processing  rate 
Pjl  =  pj2  “  •••  3  y  j  £  =*  Pj»  (Note:  We  can  relax 
assumptions  11  and  12  later  in  order  to  determine 
effects  of  individual  differences  among  resources  on 
system  performance.) 

13.  Evij  j,  <  Cjj  (maximum  capacity  constraint). 


3.0  VALIDATION  (SIMCOPE-1) 


The  SIMCOPE  facility  at  AFAMRL  was  chosen  as  a  test  case  for  illustrating  the 
application  of  (preliminary)  IAT  methods.  SIMCOPE  was  selected  for  several 
reasons: 

1)  It  approximates  the  complexity  of  a  node  in  real-world  manned 
systems  like  those  at  NORAD  MWC  (even  though  its  scenarios  are 
fictitious). 

2)  Its  operations  are  neatly  circumscribed. 

3)  It  permits  the  study  of  human  performance  within  a  controlled 
laboratory  environment  (which  approximates  a  real-world  opera¬ 
tional  setting). 

4)  The  details  of  its  design  and  operation  can  be  analyzed  and 
discussed  openly  (because  of  its  fictitious  nature). 

The  focus  of  SIMCOPE  is  on  the  Missile  Warning  Officer  (MWO),  whose  main  role 
is  to  monitor  data  to  detect  missile  launches  that  may  pose  a  possible  threat. 
Performance  predictions  relevant  for  the  validation  address  questions  of 
system  throughput  —  e.g.,  what  happens  as  input  message  arrivals  exceed 
operator  service  rates?  what  operating  strategies  best  handle  the  work 
backlog?  are  there  alternative  designs  that  can  alleviate  the  bottleneck? 

For  using  SIMCOPE  as  a  case  study  to  validate  IAT  methods,  several  notational 
formats  were  required.  These  are  reviewed  in  the  sections  that  follow. 


3.1  OPERATOR  SCENARIOS 

These  specify  the  tasks  and  task  sequences  that  MWOs  would  perform  to  carry 
out  their  responsibilities  In  a  MWC.  (Subject  instructions  were  developed 
under  the  AFAMRL  COPE  Program  for  this  purpose,  and  are  described  in  [9].) 


3.2  DATA  FLOWS  (DeMarco) 

Because  methods  like  QNT  require  explicit  information  about  data  flow,  a 
formalism  was  needed  that  would  capture  this  Information  in  SIMCOPE.  Of 
critical  interest  are  the  following  data  flow  characteristics: 

-  Data  Stores:  sites,  temporary  repositories  of  data 
(e.g. ,  computer  files,  blackboards,  operator  displays). 

-  Sources:  points  of  origin. 

-  Sinks:  points  of  destination. 

-  Processes:  transformations  that  map  input  data  to  output  data. 

-  Flows:  "pipelines"  through  which  packets  of  Information  of  known 
composition  may  flow. 


DeMarco  diagrams  (10]  provide  a  simple  syntax  and  semantics  Cor  capturing 
these  characteristics.  Figure  3-1  describes  the  components  of  a  DeMarco 
diagram;  Figure  3-2  presents  an  example  oC  how  these  diagrams  were  used  to 
portray  data  flow  in  SIMCOPG. 


SOURCE 


SINK 


FILE 


COMPONENTS 


1.  Data  flows,  represented  by  labeled  arrows;  (X.T.Z) 

2.  Processes,  represented  by  circles;  (Pj.Pj) 

3.  Files  or  Data  Stores,  represented  by  straight  lines;  FILE 

4.  Data  Sources  or  Sinks,  represented  by  boxes. 

Figure  3-1.  DeMarco  Data  Flow  Diagram  (Example) 

CONTEXT 


Figure  3-2.  DeMarco  Diagrams  Used  in  SIMC0PE  Validation  [5] 


3.3  USE  OF  FRAME  NOTATION 


Frames  were  used  to  describe  aspects  of  information  captured  in  the  data  flow 
diagrams.  This  was  done  to  facilitate  cross-referencing  of  data  elements  and 
provide  traceability  from  data  and  processes  (shown  in  DeMarco  diagrams)  back 
to  components  of  system  structure  (GOALS,  ORGANIZATIONS,  PROCESSES,  RESOURCES) 
Figure  3-3  (shown  as  Figure  2-7  presented  earlier)  is  an  example  of  a  PROCESS 
Frame  for  the  process  called  "Monitor  for  Enemy  Missile  Launch"  in  the  SIMCOPE 
context;  Figure  3-4  shows  the  hierarchical  decomposition  in  terms  of  system 
structure. 
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Figure  3-3 •  Example  of  IAT  Structural  Description  and  Process  Frame 
for  the  Process  "Monitor  for  Enemy  Missile  Launch" 
(Based  on  SIMCOPE-1  Analysis  [5]) 


MONITOR  FOR 


Figure  3-4.  Monitor  for  Eneay  Missile  Launch: 
Tree  Structure  Hierarchy 
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3.4  QNT  REPRESENTATION 


Representing  Information  about  SIMCOPE  structure  and  behavior  with  the 
formalisms  described  above  made  it  possible  to  construct  a  queuing  model 
(Figure  3-5).  Arrival  rates  and  service  rates  were  specified,  and  conditional 
service  times  estimated.  Estimates  were  then  derived  to  describe  the  time 
required  for  completing  report-processing.  Further  details  are  provided  in 
[5]  to  explain  how  the  model  was  constructed  and  the  system  performance 
parameters  derived. 


INPUT  MESSAGES: 

•  EQUIPMENT 
STATUS 

•  INTELLIGENCE 

•  SUSPECTED 
LAUNCH 
EVENTS 


Figure  3-5.  The  Queuing  Representation  of  SIMCOPE 


3.5  USE  OF  ANALYSIS  RESULTS  FOR  CHANGES/REDESIGN  OF  C3  SYSTEMS 

The  queuing  network  model  allowed  investigators  to  pose  "WHAT-IF"  questions 
about  human/ system  performance.  For  example,  one  could  examine  the  benefits 
of  adding  one  or  more  operators  (MWOs)  by  adjusting  the  structure  and  the 
appropriate  service  time  parameters  in  the  model.  In  another  case,  one  could 
examine  what  would  happen  if  an  excessive  number  of  routine  status  or 
intelligence  messages  came  in  and  overloaded  the  system  [5). 


4.0  PLANS  FOR  FY85 


4.1  Methods  Development  Strategy 

As  discussed  earlier,  much  of  the  work  done  to  date  has  focused  on  developing 
the  1AT  conceptual  framework  and  its  component  techniques  and  notations. 
Preliminary  validation  of  these  techniques  on  S1MC0PE  has  helped  IAT  methods 
developers  identify  areas  of  strength  as  well  as  needs  for  further  development. 

The  most  important  tasks  to  be  addressed  in  FY85  pertain  to  the  use  of  IAT  and 
its  application  to  real-world  C3  systems.  First,  structural  and  performance 
modeling  components  of  IAT  must  be  integrated.  Second,  guidelines  and 
procedures  must  be  developed  to  insure  applicability  of  the  methods  to  actual 
systems.*  The  sections  that  follow  summarize  the  work  required  to  satisfy 
these  needs. 


Insuring  Integration  of  Structural  and  Performance  Modeling 
Methods  must  be  developed  to  specify  the  following  activities: 

1)  Collecting  data  to  build  a  structural  model. 

2)  Carrying  out  recursive  decomposition  consistently  across  all 
dimensions  (GOALS,  ORGANIZATIONS,  PROCESSES,  RESOURCES). 

3)  Collecting  data  to  build  a  performance  model. 

4)  Using  a  consistent  syntax  and  semantics  to  represent  data  in 
matrices  and  frames. 

5)  Using  modeling  tools  such  as  QNT. 

6)  Extracting  quantitative  information  appropriate  to  answer 
"WHAT-IF”  questions. 

7)  Using  results  of  performance  analysis  to  change  or  improve  C3 
systems . 


4.2  Validation  Based  on  Real-World  C3  Systems  (NORAD  Command  Center) 

Missile  warning  processing  at  the  NORAD  Command  Center  will  provide  a  test  of 
the  validity  of  the  methodology.  This  validation  will  provide  direction  for  the 
methods  development  work  that  still  remains  to  be  completed  —  insufficiencies 
will  be  spotted  when  the  current  set  of  methods  is  exercised  in  the  field. 


In  particular,  we  would  hope  to:  (1)  determine  the  levels  of  difficulty 
associated  with  collecting  data  (#1,3  above);  and  2)  assess  the  degree  to 
which  quantitative  information  about  human  performance  can  be  obtained  by 
using  IAT  (#6,7).  Once  the  utility  of  I AT  as  a  human  engineering  technique 
has  been  demonstrated  on  real-world  systems,  we  will  be  in  a  better  position 
to  insure  that  its  use  will  improve  the  analysis  and  design  of  manned  C3 
systems. 
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